Trabalho de Implementação 2

Discipline: CMP197 - Introdução à Visão Computacional

Professor:

Authors:

Importing the necessary libraries

Setting global parameters

Task 1

Task 1A

Computing Block Matching using SSD

Task 1B

Computing Block Matching using SSD + Penalty value

Looping over windows and penalties

(Be aware: this can take awhile and consume a lot of memory!)

Comments

Considering only a pixel as reference (neighborhood of 1x1) there is a big room for ambiguity pixels on the block matching calculation. Analyzing the outcomes from tasks 1A and 1B, we can visualize this problem. Because of the ambiguity of pixels between both stereo images, the disparity map obtained ends up to be very noisy.

When we apply the match windows approach we get better results. So, as we can see in the previous images plots, as we increase the window sizes until a certain neighborhood size we get smoother disparity maps with fewer noise. However, as the window size increases we end up losing fine details, such as object boundaries.

Lastly, regarding the robust distance metric (i.e., the penalty value) which we applied in the SSD calculation of the Task 1B, we can conclude (by testing different penalty values) that it seems to be difficult to define the right setup for it (i.e., the best values of window size and penalty value to be used in order to get the best result). As we increase the penalty value, we get fewer matching mistakes (black pixels in the previous images) on the disparity maps comparing the plots with smaller penalty values or even without penalty; but, on the other hand, we also end up losing fine details, such as object boundaries.

Task 2

Task 2A

Computing Block Matching using SSD + Mean Aggregation

Computing Block Matching using SSD + Penalty value + Mean Aggregation

Task 2B

Computing Block Matching using SSD + Median Aggregation

Computing Block Matching using SSD + Penalty value + Median Aggregation

Looping over windows for aggregation

(Be aware: this can take awhile and consume a lot of memory!)

Comments

Using the Mean and Median aggregation strategies, we notice an improvement on the disparity maps (regarding the ones from Task 1), because the visual results showed smoother and more detailed images. It seems to be a good strategy to first compute the disparity map using some strategy (e.g., SSD) with a smaller window size and then using an aggregation method to finetune it (i.e., remove noises), since it improves the computation speed and return better (smoother) results.

Comparing both the Mean and Median strategies against each other, we concluded that the results returned after applying the Mean filter are smoother than the Median approach. However, the last one apparently preserves more on the fine details of the objects on the scene. Moreover, since the Mean strategy allows the usage of integral images, it seems to be a more suitable strategy for applications that require fast feedback (e.g., self-driving cars), since it can be computed in a very fast fashion and ends up using low computational resources.

We also tested different window sizes values, using odd numbers ranging from 1 to 11. Analyzing the visual outcomes presented on the previous plots, we concluded that the best results were obtained using window sizes and kernel sizes (for the Mean and Median filters) of 5x5 and 7x7 (i.e., intermediate window sizes). The window size of 11x11 also presented good results, but from this neighborhood size and beyond, the disparity maps start to lose details, such as object boundaries and textures.